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1 ABSTRACT 

Modem  airborne  surveillance  systems  have  to  cope  with  an  immense  number  of  inputs  from  real  wanted  and 
unwanted  ground,  maritime  and/or  airborne  targets  as  well  as  correlated  clutter.  This  causes  significant 
challenges  for  tracking,  classification,  and  identification  of  the  detected  objects.  A state  of  the  art  multi- 
sensor integration  (MSI)  system  calls  for  real-time  integration  of  all  available  information  pertaining  to  a real 
world  object,  in  particular,  geometric,  kinematic,  and  signature  data.  The  MSI  system  provide  by 
DaimlerChrysler  Aerospace  yields  improved  tracking  quality  and  performs  automatic  identification  of 
targets. 

The  multi-sensor  integration  system  processes  data  of  various  sensors,  e.g.,  primary  surveillance  radar, 
secondary  surveillance  radar  (IFF),  data  of  passive  electronic  support  measurements  (ESM),  acoustic  sensor 
systems,  crosstold  data  of  Link-1 1 and  Link-16  etc..  Based  upon  these  data,  the  system  will  perform  two 
tasks:  Multi-sensor  tracking  and  multi-sensor  identification. 

New  tracks  will  be  initiated  automatically  based  upon  data  input  from  active  or  passive  sensors,  respectively. 
State  of  the  art  multi-model  technology  is  used.  It  guarantees  optimal  track  stability  under  various  maneuver 
conditions.  In  the  correlation  function,  either  an  improved  nearest  neighbor  algorithm  or  the  advanced  multi- 
hypothesis concepts  are  applied..  ESM  reports  from  tactical  data  links  and  specific  sensors  are  used  to 
perform  multi-sensor  cooperative  passive  tracking. 

The  multi-sensor  identification  system  (MSI)  is  capable  to  identify  air,  surface,  and  ground  tracks.  It  will 
operate  fully  automatically  using  all  available  data  from  all  sensors  as  well  as  derived  and  background 
information  depending  on  the  confidence  which  is  associated  with  the  particular  source.  The  system  is  able  to 
handle  various  identification  schemes  in  parallel.  The  automatic  identification  process  will  evaluate  all 
available  identification  information  for  every  MSI  track  using  “identification  indicators”.  The  identification 
indicators  will  be  combined  to  yield  a track  identity  using  an  artificial  intelligence  (AI)  supported 
combination  process.  Both,  the  MSI  tracking  and  MSI  identification  functions  can  be  controlled  by  mission 
data  which  are  provided  at  mission  startup  or  at  any  time  during  the  mission. 


2 THE  MULTI-SENSOR  INTEGRATION  SYSTEM 
2.1  MSI  System  Overview 

The  ultimate  goal  of  all  multi-sensor  integration  systems  is  to  generate  a precise  picture  of  the  according  real 
world  scenario.  The  system  has  to  merge  all  available  inputs  that  pertain  to  one  target  to  only  one  object  in 
the  picture  on  the  display.  This  picture  must  contain  only  real  and  no  false  targets,  all  correctly  identified  at 
the  correct  locations.  This  goal  can  only  be  achieved  by  a careful  harmonization  of  the  sensor  integration 
software  with  the  high  performance  of  the  contributing  sensors.  The  term  Multi-Sensor  Integration  comprises 
essentially  two  distinct  functions,  which  are  Multi-Sensor  Tracking  including  the  correlation  function  and  the 
Multi-Sensor  Identification. 


Paper  presented  at  the  RTO  SCI  Symposium  on  “System  Concepts  for  Integrated  Air  Defense  of  Multinational 
Mobile  Crisis  Reaction  Forces  ”,  held  in  Valencia,  Spain,  22-24  May  2000,  and  published  in  RTO  MP-063. 


10-2 


Our  MSI  will  provide  multi  sensor  tracking  and  identification  functionality.  The  MSICP  (Multi-Sensor 
Integration  Computer  Program)  will  consist  of  the  following  parts: 

• MSI  Tracking 

• MSI  Identification 

• MSI  Manager 


The  MSI  Tracking  and  Identification  functions  provide  the  basic  operational  functionality  of  the  MSICP.  The 
MSI  Manager  includes  functionality  mainly  in  the  area  of  control  and  monitoring,  communication, 
redundancy  support  and  test  and  maintenance  provisions  for  the  MSICP. 

2.2  The  Multi-Sensor  Tracking  System 

The  Multi-Sensor  Tracking  has  the  capability  to  use  sensor  inputs  from  a wide  variety  of  different  sensors 
which  are  the  primary  radar,  the  SSR/1FF  sensor,  etc.,  the  ESM  system  and  crosstold  sensor  inputs  via  the 
various  tactical  data  links.  It  is  based  upon  Multi-Model  technology  which  guarantees  in  the  various 
maneuver  conditions  optimal  track  stability  and  continuity.  New  tracks  will  be  initiated  automatically  either 
based  on  data  inputs  from  active  radars  or  from  passive  sensors  only.  The  tracking  process  has  the  capability 
to  perform  self-triangulation  based  on  passive  strobes  from  onboard  sensors.  Additionally,  ESM  reports  from 
tactical  data  links  are  used  together  with  reports  from  onboard  sensors  to  perform  multi-sensor  cooperative 
passive  tracking.  Deghosting  is  done  automatically. 

The  functions  of  the  Tracking  system  are  grouped  in  the  following  categories  (see  Figure  2): 

• Data  preprocessing 

• Correlation/Association 

• Track  Update 

• Track  List  Management 

• Post  processing 

The  preprocessing  comprises  functions  like  ordering  of  measurement  data  in  a timeline  related  sequence, 
performing  coordinate  system  transformation,  etc..  In  this  processing  bias  compensation  is  performed  for  all 
input  sensors. 

In  the  correlation  function,  either  the  classical  nearest  neighbor  algorithm  or  the  advanced  Multi-FIypothesis 
concepts  are  applied.  Which  concept  will  be  applied  is  dependent  on  the  actual  complexity  of  the  scenario  in 
the  vicinity  of  the  current  measurement/track  position.  Adaptive  selection  of  the  most  suitable  correlation 
algorithm  will  significantly  reduce  processor  load.  In  doing  so,  maximum  track  continuity  will  be  provided 
under  all  target  conditions,  and  hence,  track  identification  will  be  preserved  through  complicated  target  track 
crossing  situations.  Furthermore,  ESM-based  signature  data  and  cross  correlation  between  ESM-  and 
kinematic  data  will  also  be  used  for  the  final  report/track  pairing.  The  operator  can  at  any  time  either  prohibit 
a selected  correlation  or  enforce  a correlation. 

The  track  update  function  is  executed  for  each  track  which  has  been  associated  to  a target  report.  Depending 
on  the  type  of  the  target  report  either  a kinematic  and/or  a attribute  update  is  performed  for  every  model 
which  is  currently  applicable  to  the  track.  Kinematic  updating  of  active  tracks  is  done  by  means  of  a Kalman 
component  filter,  whose  peculiar  feature  is  that  updating  of  the  state  vector  is  performed  for  each 
measurement  component  separately.  A major  advantage  of  this  Kalman  component  filter  is  that  even  under 
jammed  or  distorted  conditions  all  available  useful  measurement  information  will  be  utilized.  Attribute 
updating  is  performed  by  keeping  record  of  attribute  data  of  target  reports  incorporated  into  the  track. 
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The  Track  List  Management  contains  functions  like  track  initiation,  track  deletion,  track  prediction  and  bias 
error  estimation.  Track  initiation  is  performed  on  target  reports  for  which  no  correlation  was  found.  They  will 
be  initiated  as  tracks  and  labeled  as  potential  track.  Initiation  of  tracks  will  not  only  be  performed  on  sensor 
data  from  active  sensors  but  also  on  data  from  passive  sensors.  Track  deletion  will  be  applied  to  tracks, 
whose  quality  figure  recedes  below  a certain  threshold.  These  deletion  thresholds  automatically  adapt  to 
environment  parameters  and  to  the  different  track  states.  Track  prediction  is  performed  for  the  time  when  the 
next  sensor  update  is  expected.  The  predicted  position  is  input  to  the  correlation  function  the  next  time.  The 
bias  error  estimation  function  analyzes  measurement  residuals  of  each  sensor  with  respect  to  the  system  hack. 
In  order  to  maximize  overall  track  quality  and  accuracy,  possible  offsets  between  all  sensors  are  monitored 
and  estimated  continuously  in  background.  If  necessary,  observed  offsets  are  compensated  automatically  by 
an  adaptive  logic. 

In  the  post  processing,  coordinate  system  transformations  are  performed  to  adapt  the  tracker  to  the  external 
system.  Finally,  data  is  converted  to  external  data  formats. 

In  summary,  the  functions  performed  by  the  MSI-Tracking  are: 

• processing  of  input  data  from  different  types  of  sensors  - Primary  Radar,  IFF,  ESM,  Links 

• merge  active  and  passive  tracking  (both  cooperative  and  self  triangulation)  including  sophisticated 
correlation  and  association  logic.  This  makes  use  of  geometric  data  and  attributes.  The  correlation  and 
association  logic  uses  rules  about  identity  indications  from  ID  sources 

• update  each  measurement  component  (Range,  AZ,  EL,  Range  Rate)  separately,  using  a special  form  of 
Kalman  Filter  algorithm 

• automatically  associate  of  signature  and  attribute  parameters  (IFF,  ESM,  ECM,  Link  data)  to  tracks 

• update  attributes  (IFF  codes,  ESM  attributes,  ECM  attributes) 

• compensate  ownship  motion 

• automatic  track  initiation  and  track  drop 

• automatic  maneuver  detection 

• measurement  dropout  coasting 

• automatic  detection  & compensation  of  bias  (registration)  errors 

• processing  of  operator  initiated  hack  commands 

• determination  of  target  environment  (air,  ground,  surface)  based  on  kinematics  data 


Upon  reception  of  system  control  commands  issued  automatically  by  the  MSI  monitoring  & control  as  part  of 
MSI  management  SW,  the  tracking  adapts  its  functions  automatically  to  graceful  degradation  measures  in 
order  to  prevent  uncontrolled  program  behavior  in  case  of  special  conditions  (e.g.  overload).  Additionally,  the 
MSI-Tracking  function  monitors  sensor  input  data  for  plausibility  in  order  to  generate  inputs  for  error 
logging. 

2.3  The  Multi-Sensor  Identification  System 

The  MSI  Identification  function  has  the  capability  to  identify  air,  surface,  and  ground  tracks.  It  will  operate 
fully  automatic  and  uses  all  available  data  of  the  available  sensors  as  well  as  derived  information  and 
background  information  dependent  on  the  confidence  which  can  be  given  to  the  infoimation  sources.  The 
system  can  handle  different  identification  schemes  in  parallel  (e.g.,  suited  for  either  peace,  tension,  war  states, 
contingency  missions  etc.).  The  identification  schema  is  loaded  as  data  during  system  initialization. 
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High  flexibility  in  the  ID  process  is  achieved  by  providing  the  operator  with  several  means  to  adapt  the 
identification  function  to  mission  specific  needs.  Some  of  these  means  are: 

• modification  of  the  set  of  information  to  be  used  for  identification, 

• modification  of  mission  data 

• overriding  of  identification  results. 


The  functions  of  the  multi-sensor  identification  are  based  upon  artificial  intelligence  concepts,  which  use  a 
rule  based  artificial  intelligence  system.  This  rule-based  system  was  developed  by  DASA  ASD  with  the  focus 
to  be  used  in  operational  systems  and  give  responses  with  minimum  delay. 

The  MSI  Identification  function  does  provide  the  capability  to  assign  track  identities  based  on  integrated 
sensor,  communications  and  operator  provided  data.  ID  related  information  as  for  example  conformance  of 
tracks  to  flight  plans,  or  platform  identification  by  origin  (IDBO),  which  on  its  own  may  not  provide 
conclusive  ID  are  combined  in  order  to  extract  maximal  benefit  from  the  available  sources  of  data.  Conflict 
resolution  is  performed  in  case  of  contradictory  data.  Therefore,  the  MSI-  Identification  function  will  make 
maximum  use  of  the  available  information  to  provide  its  results. 

The  MSI  Identification  function  uses  the  information  from  different  sources  to  determine  the  identity  of  MSI 
tracks.  These  sources  are: 

• 2D-position,  altitude,  speed  and  heading,  their  error  values,  environment  category  (air,  surface,  ground) 
and  time  of  track  update  of  the  targets  from  MSI  Track  File 

• ID  related  data  from  data  links 

• Mission  data 

All  available  information  for  the  MSI  Identification  process  is  used  in  a most  effective  way  to  derive  a unique 
target  identification  and  classification  for  the  environment  categories  air,  surface,  and  ground.  The 
identification  is  done  automatically  or  by  operator  input.  The  manual  ID  assignment  has  priority  over 
automatically  determined  ID.  In  case  of  manual  ID  assignment  automatic  ID  determination  will  continue  and 
MSI  ID  will  report  detected  differences  between  automatic  identification  based  on  available  information  and 
the  manually  assigned  ID. 

The  automatic  identification  process  will  evaluate  all  available  identification  information  for  every  MSI  track 
using  “identification  indicators”.  These  identification  indicators  will  be  combined  into  a track  identity  by  an 
artificial  intelligence  (AI)  supported  combination  process.  Conflicts  in  the  available  identification 
information  will  be  detected.  The  automatic  system  will  provide  conflict  resolution  functionality.  In  cases 
where  conflict  resolution  is  not  possible  operator  alerts  will  be  generated  automatically.  Possible  ID  conflicts 
with  ID  data  from  the  data  links  are  indicated  to  the  operator.  In  case  of  detected  severe  ID-changes  (e.g. 
from  FRIEND  to  HOSTILE  or  vice  versa)  a manual  ID  assignment  will  be  requested. 

The  MSI  Identification  function  will  provide  rationale  for  its  results  to  the  operator.  The  complete  MSI 
Identification  process  will  be  controlled  in  a flexible  way  by  operator  changeable  “Adaptation  Data”. 

The  incoming  (live)  sensor  and  communication  data  may  be  mixed  with  simulated  data  in  order  to  train 
operators  during  real  mission  flights. 

Upon  reception  of  system  control  commands  the  MSI  Identification  adapts  its  functionality  automatically  to 
graceful  degradation  measures  to  prevent  uncontrolled  program  behavior  in  case  of  special  conditions  (e.g. 
overload). 
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2.4  MSI  Manager 

The  MSI  Manager  function  will  integrate  the  MSI  Tracking  and  Identification  functions.  It  will  provide  all 
necessary  communication  mechanisms  for  MSI  Tracking  and  MSI  Identification  based  on  the  Real  Time 
Communication  middleware. 

The  MSICP  application  software  will  use  the  Real  Time  Communication  Layer  and  MC/MSIC  System 
Software  layers  and  provide  CORBA  2.0  compliant  interfaces  for  external  communication  and 
communication  between  major  internal  software  components  (cf.  Figure  1). 

An  overall  MSICP  system  control  will  be  provided  by  the  MSI  Manager  including  system  startup  and 
shutdown  mechanisms  as  well  as  the  overall  control  of  operational  modes  of  the  MSICP.  The  MSI  Manager 
will  also  provide  an  overall  system  monitoring  function.  This  function  will  provide  all  required  MSICP 
system  status  data  in  regular  manner  to  the  MSICP  external  interfaces. 

The  MSICP  will  be  responsible  for  redundancy  support  of  MSI  Tracking  and  MSI  Identification. 

The  MSI  Manager  will  be  responsible  for  the  overall  computer  load  management  within  the  MSICP.  It  will 
measure  computer  load,  detect  overload  situations  and  activate  the  load  adaptation  functionality  of  the  MSI 
Tracking  and  Identification  function  to  prevent  unpredictable  behavior. 

Within  the  MSI  Manager  and  within  MSI  Tracking  and  Identification  provisions  for  test  functionality  will  be 
included,  which  can  be  activated  by  the  MSI  Software  Test  Bed.  These  provisions  will  be  used  during  system 
testing  and  integration  and  may  be  used  for  maintenance  activities  in  the  software  life  cycle.  These  provisions 
will  support: 

• recording  of  external  and  internal  MSICP  interfaces 

• logging  of  MSICP  internal  data  for  test  and  maintenance  purposes. 

The  MSI  Software  Test  Bed  will  include  the  functionality  to  make  use  of  these  provisions  including  the 
external  interfaces. 


3 SYSTEM  TEST 

During  Test  and  Development  the  MSI  Software  Test  Bed  provided  by  DAS  A Airborne  Systems  will  include 
all  functionality  needed  to  operate  the  MSICP  in  a test  environment.  It  will  additionally  provide  simulations 
of  Mission  Computer  functionality  as  needed  to  run  the  MSICP  in  a test  bed.  The  MSI  Software  Test  Bed 
will  simulate  interfaces  to  the  MSICP  compatible  to  the  MCP  real  time  interfaces.  Parts  of  this  MSI  Software 
testbed  may  also  be  used  for  maintenance  puiposes  during  the  operational  usage  of  the  MSICP.  Functionality 
and  performance  of  the  MSICP  will  be  verified  by  a two  stage  approach.  First,  testing  is  performed  via 
computer  models.  The  Test  and  Development  Support  software  contains  a variety  of  representative  scenarios 
and  generates  the  inputs  of  all  relevant  sources.  It  also  considers  the  sensor  behavior  that  affects  detection 
performance  and  accuracy  of  the  measurements.  The  Test  and  Development  Support  Software  (TADS)  will 
include  all  features  needed  to  generate  the  necessary  data,  and  do  the  analysis  of  results.  Second,  the  MSICP 
is  tested  by  using  recorded  life  data.  The  evaluation  of  the  results  is  also  performed  by  the  TADS. 


In  the  final  qualification  test  run,  the  configuration  data  of  the  multi-sensor  tracking  and  identification 
function  will  be  optimized  during  real  flight  tests.  This  will  finally  verify  the  functions  and  the  performance 
of  the  MSICP. 
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Figure  1:  Software  Layers  (Test  Configuration) 


4 SUMMARY 

In  this  article  we  have  presented  a brief  overview  of  the  DaimlerChrysler  Multi- Sensor  Integration  System.  It 
explains  the  requirements  for  the  new  system  and  its  performance  drivers  as  well  as  the  concepts  which  were 
applied  in  the  system  design  in  order  to  match  the  practical  requirements.  Especially  in  the  identification 
function  a new  dimension  of  flexibility  is  implemented  to  enable  operators  to  cope  with  the  changing  real 
world  situation. 


Figure  2:  MSI  Tracker 


